• Now you need to know how to do ITEM ANALYSIS
  
  
  --> looks complicated at first glance, but actually quite simple
  
  -->even I can do this and I'm a mathematical idiot

  

• Talking about norm-referenced, objective tests
  
  
• mostly multiple-choice but same principals for true-false, matching and short answer
  
  
• by analyzing results you can refine your testing

1. Fix marks for current class that just wrote the test
   • find flaws in the test so that you can adjust the mark before return to students
     
     
   • can find questions with two right answers, or that were too hard, etc., that you may want to drop from the exam
     
     
     • even had to do that occasionally on Diploma exams, even after 36 months in development, maybe 20 different reviewers, extensive field tests, still occasionally have a question whose problems only become apparent after you give the test
       
       
     • more common on classroom tests -- but instead of getting defensive, or making these decisions at random on basis of which of your students can argue with you, do it scientifically

2. More diagnostic information on students
   • another immediate payoff of item analysis

   Classroom level:

   • will tell which questions they were are all guessing on, or if you find a questions which most of them found very difficult, you can reteach that concept
     
     
   • CAN do item analysis on pretests to:
     
     • so if you find a question they all got right, don't waste more time on this area
       
       
     • find the wrong answers they are choosing to identify common misconceptions
       
       
     • can't tell this just from score on total test, or class average

   Individual level:

   • isolate specific errors this child made
     
     
   • after you've planned these tests, written perfect questions, and now analyzed the results, you're going to know more about these kids than they know themselves

3. Build future tests, revise test items to make them better
   • REALLY pays off second time you teach the same course
     
     
     • by now you know how much work writing good questions is
       
       
     • studies have shown us that it is FIVE times faster to revise items that didn't work, using item analysis, than trying to replace it with a completely new question
       
       
     • new item which would just have new problems anyway
       
       --> this way you eventually get perfect items, the envy of your neighbours

   • SHOULD NOT REUSE WHOLE TESTS --> diagnostic teaching means that you are responding to needs of your students, so after a few years you build up a bank of test items you can custom make tests for your class
     
     
     • know what class average will be before you even give the test because you will know approximately how difficult each item is before you use it;
       
       
     • can spread difficulty levels across your blueprint too...

4. Part of your continuing professional development
   • doing the occasional item analysis will help teach you how to become a better test writer
     
     
   • and you're also documenting just how good your evaluation is
     
     
   • useful for dealing with parents or principals if there's ever a dispute
     
     
   • once you start bringing out all these impressive looking stats parents and administrators will believe that maybe you do know what you're talking about when you fail students...
     
     
   • parent says, I think your "question stinks",
     
     well, "according to the item analysis, this question appears to have worked well -- it's your son that stinks"
     
     (just kidding! --actually, face validity takes priority over stats any day!)
     
     
   • and if the analysis shows that the question does stink, you've already dropped it before you've handed it back to the student, let alone the parent seeing it...

5. Before and After Pictures
   • long term payoff
     
     
   • collect this data over ten years, not only get great item bank, but if you change how you teach the course, you can find out if innovation is working
     
     
   • if you have a strong class (as compared to provincial baseline) but they do badly on same test you used five years ago, the new textbook stinks.

ITEM ANALYSIS is one area where even a lot of otherwise very good classroom teachers fall down

• they think they're doing a good job; they think they've doing good evaluation, but without doing item analysis, they can't really know
  
  
• part of being a professional is going beyond the illusion of doing a good job to finding out whether you really are
  
  
• but something just a lot of teachers don't know HOW to do
  
  
• do it indirectly when kids argue with them...wait for complaints from students, student's parents and maybe other teachers...

ON THE OTHER HAND....

• I do realize that I am advocating here more work for you in the short term, but, it will pay off in the long term

But realistically:

*Probably only doing it for your most important tests

• end of unit tests, final exams --> summative evaluation
  
  
• especially if you're using common exams with other teachers
  
  
• common exams give you bigger sample to work with, which is good
  
  
• makes sure that questions other teacher wrote are working for YOUR class
  
  
• maybe they taught different stuff in a different way
  
  
• impress the daylights out of your colleagues
  
  

• item analysis is not an end in itself,
  
  
• no point unless you use it to revise items,
  
  
• and help students on basis of information you get out of it

I also find that, if you get into it, it is kind of fascinating. When stats turn out well, it's objective, external validation of your work. When stats turn out differently than you expect, it becomes a detective mystery as you figure out what went wrong.
But you'll have to take my word on this until you try it on your own stuff.